LSOS: Line-search second-order stochastic optimization methods for nonconvex finite sums

We develop a line-search second-order algorithmic framework for minimizing finite sums. do not make any convexity assumptions, but require the terms of sum to be continuously differentiable and have Lipschitz-continuous gradients. The methods fitting into this combine line searches suitably decaying step lengths. A key issue is two-step sampling at each iteration, which allows us control error present in procedure. Stationarity limit points proved almost-sure sense, while convergence sequence approximations solution holds with additional hypothesis that functions are strongly convex. Numerical experiments, including comparisons state-of-the art stochastic optimization methods, show efficiency our approach.